A survey of cross-lingual word embedding models

Cross-lingual representations of words enable us to reason about word meaning in multilingual contexts and are a key facilitator of cross-lingual transfer when developing natural language processing models for low-resource languages. In this survey, we provide a comprehensive typology of cross-lingual word embedding models. We compare their data requirements and objective functions. The recurring theme of the survey is that many of the models presented in the literature optimize for the same objectives, and that seemingly different models are often equivalent, modulo optimization strategies, hyper-parameters, and such. We also discuss the different ways cross-lingual word embeddings are evaluated, as well as future challenges and research horizons.</jats:p

On the limitations of unsupervised bilingual dictionary induction

Unsupervised machine translation---i.e., not assuming any cross-lingual supervision signal, whether a dictionary, translations, or comparable corpora---seems impossible, but nevertheless, Conneau et al. (2018) recently proposed a fully unsupervised machine translation (MT) model. The model relies heavily on an adversarial, unsupervised alignment of word embedding spaces for bilingual dictionary induction, which we examine here. Our results identify the limitations of current unsupervised MT: unsupervised bilingual dictionary induction performs much worse on morphologically rich languages that are not dependent marking, when monolingual corpora from different domains or different embedding algorithms are used. We show that a simple trick, exploiting a weak supervision signal from identical words, enables more robust induction, and establish a near-perfect correlation between unsupervised bilingual dictionary induction performance and a previously unexplored graph similarity metric

Sequence classification with human attention

Learning attention functions requires large volumes of data, but many NLP tasks simulate human behavior, and in this paper, we show that human attention really does provide a good inductive bias on many attention functions in NLP. Specifically, we use estimated human attention derived from eye-tracking corpora to regularize attention functions in recurrent neural networks. We show substantial improvements across a range of tasks, including sentiment analysis, grammatical error detection, and detection of abusive language

Analogy Training Multilingual Encoders

Language encoders encode words and phrases in ways that capture their local semantic relatedness, but are known to be globally inconsistent. Global inconsistency can seemingly be corrected for, in part, by leveraging signals from knowledge bases, but previous results are partial and limited to monolingual English encoders. We extract a large-scale multilingual, multi-word analogy dataset from Wikidata for diagnosing and correcting for global inconsistencies and implement a four-way Siamese BERT architecture for grounding multilingual BERT (mBERT) in Wikidata through analogy training. We show that analogy training not only improves the global consistency of mBERT, as well as the isomorphism of language-specific subspaces, but also leads to significant gains on downstream tasks such as bilingual dictionary induction and sentence retrieval

Discerning the role of polymicrobial biofilms in the ascent, prevalence, and extent of heteroresistance in clinical practice

Antimicrobial therapy is facing a worrisome and underappreciated challenge, the phenomenon of heteroresistance (HR). HR has been gradually documented in clinically relevant pathogens (e.g. Pseudomonas aeruginosa, Staphylococcus aureus, Burkholderia spp., Acinetobacter baumannii, Klebsiella pneumoniae, Candida spp.) towards several drugs and is believed to complicate the clinical picture of chronic infections. This type of infections are typically mediated by polymicrobial biofilms, wherein microorganisms inherently display a wide range of physiological states, distinct metabolic pathways, diverging refractory levels of stress responses, and a complex network of chemical signals exchange. This review aims to provide an overview on the relevance, prevalence, and implications of HR in clinical settings. Firstly, related terminologies (e.g. resistance, tolerance, persistence), sometimes misunderstood and overlapped, were clarified. Factors generating misleading HR definitions were also uncovered. Secondly, the recent HR incidences reported in clinically relevant pathogens towards different antimicrobials were annotated. The potential mechanisms underlying such occurrences were further elucidated. Finally, the link between HR and biofilms was discussed. The focus was to recognize the presence of heterogeneous levels of resistance within most biofilms, as well as the relevance of polymicrobial biofilms in chronic infectious diseases and their role in resistance spreading. These topics were subject of a critical appraisal, gaining insights into the ascending clinical implications of HR in antimicrobial resistance spreading, which could ultimately help designing effective therapeutic options.This work was supported by the Portuguese Foundation for Science Technology (FCT) under the scope of the strategic funding of UID/BIO/04469/2020 unit BioTecNorte operation [NORTE-01-0145-FEDER-000004] funded by the European Regional Development Fund under the scope of Norte2020–Programa Operacional Regional do Norte. The authors also acknowledge COMPETE2020 FCT for the project POCI-01-0145-FEDER-029,841 and for the Scientific Employment Stimulus 2017 grant [CEECIND/01507/2017] (A. M. Sousa).info:eu-repo/semantics/publishedVersio

Pakistanis living in Oslo have lower serum 1,25-dihydroxyvitamin D levels but higher serum ionized calcium levels compared with ethnic Norwegians. The Oslo Health Study

Background Persons of Pakistani origin living in Oslo have a much higher prevalence of vitamin D deficiency and secondary hyperparathyroidism but similar bone mineral density compared with ethnic Norwegians. Our objective was to investigate whether Pakistani immigrants living in Oslo have an altered vitamin D metabolism by means of compensatory higher serum levels of 1,25-dihydroxyvitamin D (s-1,25(OH)2D) compared with ethnic Norwegians; and whether serum levels of ionized calcium (s-Ca2+) differ between Pakistanis and Norwegians. Methods In a cross-sectional, population-based study venous serum samples were drawn from 94 Pakistani men and 67 Pakistani women aged 30–60 years, and 290 Norwegian men and 270 Norwegian women aged 45–60 years; in total 721 subjects. Results Pakistanis had lower s-1,25(OH)2D compared with Norwegians (p < 0.001). Age- and gender adjusted mean (95% CI) levels were 93 (86, 99) pmol/l in Pakistanis and 123 (120, 126) pmol/l in Norwegians, p < 0.001. The difference persisted after controlling for body mass index. There was a positive relation between serum 25-hydroxyvitamin D (s-25(OH)D) and s-1,25(OH)2D in both groups. S-Ca2+ was higher in Pakistanis; age-adjusted mean (95% CI) levels were 1.28 (1.27, 1.28) mmol/l in Pakistanis and 1.26 (1.26, 1.26) mmol/l in Norwegians, p < 0.001. In both groups, s-Ca2+ was inversely correlated to serum intact parathyroid hormone levels (s-iPTH). For any s-iPTH, s-Ca2+ was higher in Pakistanis, also when controlling for age. Conclusion Community-dwelling Pakistanis in Oslo with low vitamin D status and secondary hyperparathyroidism have lower s-1,25(OH)2D compared with ethnic Norwegians. However, the Pakistanis have higher s-Ca2+. The cause of the higher s-Ca2+ in Pakistanis in spite of their higher iPTH remains unclear